Luttinger Liquid Physics and Spin-Flip Scattering on Helical Edges 
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We investigate electronic correlation effects on edge states of quantum spin-Hall insulators within 
the Kane-Mele-Hubbard model by means of quantum Monte Carlo simulations. Given the U{1) spin 
symmetry and time-reversal invariance, the low-energy theory is the helical Tomanaga-Luttinger 
model, with forward scattering only. For weak to intermediate interactions, this model correctly 
describes equal-time spin and charge correlations, including their doping dependence. As apparent 
from the Drude weight, bulk states become relevant in the presence of electron-electron interactions, 
rendering the forward-scattering model incomplete. Strong correlations give rise to slowly decaying 
transverse spin fluctuations, and inelastic spin-flip scattering strongly modifies the single-particle 
spectrum, leading to graphene-Iike edge state signatures. The helical Tomanaga-Luttinger model is 
completely valid only asymptotically in the weak-coupling limit. 
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Introduction. — A unique feature of quantum spin Hall 
insulators (QSHIs), or two-dimensional (2D) topological 
insulators, are metallic edge states with remarkable prop- 
erties [T]. Contrary to chiral quantum Hall edge states, 
QSHI edge states are helical, so that electrons with oppo- 
site spin propagate in opposite directions. Due to time- 
reversal invariance (TRI) , the helical edge states are pro- 
tected against disorder and single-particle backscattering 
[U [2] . They are also holographic in the sense that they 
exist only as edges of 2D systems [5], and can therefore 
not be completely separated from the bulk. The number 
of pairs of edge states is directly related to the second 
Chern number or Z2 invariant [1]. For a review of topo- 
logical insulators, see Ref. [3]. 

Quantum fluctuations play a significant role for 2D 
topological insulators. In particular, the one-dimensional 
(ID) edges have no well-defined quasiparticle excita- 
tions, and are usually described using the framework of 
bosonization or Luttinger liquid (LL) theory, which be- 
comes particularly simple in the presence of TRI and 
U{1) spin symmetry. In this case [referred to in the fol- 
lowing as the helical Tomanaga-Luttinger (HTL) model] , 
only forward scattering is possible A priori, such a 
theory is only valid at low energies. Nevertheless, for 
metallic ID systems, LL theory provides a complete low- 
energy description even for strong interactions. For heli- 
cal edge states, the presence of bulk states is intimately 
connected with the topological character of the system. 
Strong interactions on or beyond the size of the bulk band 
gap can give rise to a substantial mixing of the different 
energy scales, and may explain deviations of, e.g., the 
experimentally measured conductance [S] from expecta- 
tions based on a low-energy description. Bulk effects fall 
outside the regime of bosonization, and require a model 
which captures all relevant energy scales. A comparison 
of the spectral properties of helical, spiral and standard 
Luttinger liquids has been given in Ref. [6]. 

The Kane-Mele (KM) model [1] of noninteracting elec- 
trons on the honeycomb lattice with spin-orbit (SO) cou- 



pling A is a theoretical framework to study Z2 QSHIs. 
For small enough Rashba coupling, the ground state for 
A > is a topological band insulator (TBI). The addi- 
tion of a Hubbard interaction term (KMH model) per- 
mits to study a strongly correlated TBI [7^, although it 
completely detaches the KM model from its original moti- 
vation by graphene [5] . The electron-electron interaction 
leads to a complex and rich many-body problem, which 
includes a quantum spin liquid phase, and a magnetic 
transition at large Hubbard U [71 (SHU] • 

In this Rapid Communication, using large-scale quan- 
tum Monte Carlo (QMC) simulations of a previously in- 
troduced effective model |9j , we provide a comprehensive 
assessment of the validity of the HTL model. While con- 
firming the interaction and doping dependence predicted 
for a helical liquid in the weakly interacting limit, we find 
significant deviations with increasing correlations which 
are beyond the usual low-energy description and had re- 
mained unnoticed in previous numerical work [HI 1111 112) . 
Additional features in the single-particle spectrum are ex- 
plained by inelastic spin-flip scattering arising from mag- 
netic fluctuations at the edge, driven by strong electronic 
correlations. The interaction-driven mixing of multiple 
energy scales is more subtle than the invalidation of the 
HTL model due to the breaking of TRI, for example by 
means of strong bulk interactions [71 [HI HH [H] which 
destroy the topological character, or by sufficient renor- 
malization of the LL parameter in the presence of Rashba 
coupling [2]. 

Model. — The phase diagram of the KMH model in 
the X-U plane is known from exact QMC simulations 
[21 [m [Tl] ; many of its overall features are also captured 
by approximate methods [3 UHl HI [13] . For C//t < 3 (t 
being the hopping integral), the ground state is a TBI 
with helical edges for any A > 0. The TBI with repulsive 
U > Q IS adiabatically connected to C/ = [HI E] , sug- 
gesting that bulk interactions are of minor importance in 
the TBI phase. Based on this result and the fact that 
the edge states are exponentially localized at the edge 
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[S], we have previously proposed an effective model for 
the helical edge states (which exist throughout the TBI 
phase) with Hubbard interaction only at one zigzag edge 
of a semi-infinite honeycomb ribbon. It is defined by the 
action |9j 

S = - J2 If dTdr'cUr)G-,-\r-r',r^r')c^,^iT') 
+ uY,fdT [nr^ir) - i] [n,,;(T) - \] , (1) 

J. JQ 

where r numbers sites on the zigzag edge, and Gq is the 
Green's function of the KM model through which the 
noninteracting bulk is taken into account. 

Method. — Equation ([T]) corresponds to a ID prob- 
lem with a bath that can be solved exactly using the 
continuous-time QMC (CTQMC) method P[TS] on large 
systems at low temperatures. Two crucial methodologi- 
cal developments compared to Ref. IHIare the extension to 
a projective zero-temperature scheme with projection pa- 
rameter 9, and to grand-canonical simulations away from 
half filling (n — 1). These advances allow for a quantita- 
tive test of LL theory. We use the ribbon geometry of [3] , 
with dimensions L x U {L' — 64), and periodic (open) 
boundaries in the x (y) direction. The hopping integral 
and lattice constant are set to 1. The ratio U/X of the 
remaining two parameters controls the degree of correla- 
tions. To study strong interactions inside the TBI phase 
of the KMH model, we take [/ = 2 and vary A p[T4]. 

Results. — For a zigzag ribbon, the spectral function 
A„{k,uj) = -n-HmG^{k,uj) of the [/ = KM model 
features a bulk energy gap Ago ^ 3\/3A at the Dirac 
points, and a pair of helical edge states with Fermi ve- 
locity vf ~ 2A crossing at fcp = tt (for half filling) 
[see also Fig. |3ja)]. The right (left) movers have spin 
up (down), and TRI implies A^{k,uj) = A^{-k,uj). The 
degeneracy at /c = tt is protected by Kramers' theorem. 

Correlation effects on energy scales much smaller than 
the bulk gap can be studied using bosonization. In the 
continuum limit, the fermion operator becomes 5'o-(?') = 
e''=='^7^<,(a;)^^,^ + e-''*'F^/:^(r)(5^,;. For a Hubbard in- 
teraction, the bosonized Hamiltonian (the HTL model) 
reads [2l[l6l[l7] 



H ^ 



8tt 



dx 



d^e+{x)f + K[dj-{x)\' 



(2) 



Here —dxd±{x)/2T: — 
and Pl(x) being the 
movers, respectively, v 
^{VF - U/27r)/{vF 



density 



Pl{x), with pr{x) 
of right and left 
and 



(C//27r)2 and K = 
Single-particle spin-flip 



U/2t:). 

scattering between Kramers degenerate states is blocked 
by TRI [U [21 [T7]- Umklapp processes such as 
e^'^^^^TZ'^ {x)'R) {x -\- a)C{x)C{x + a) are generally allowed 
at half filling since 4A:f = 47r, but are forbidden here due 
to spin-z conservation reflected in the U{1) spin symme- 
try of the KMH model. Hence, only forward scattering 
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FIG. 1: (Color online) Projective CTQMC results for the 
effective model ([T]). (a) Real-space correlations S^^(r = L/2) 
as a function of system size L at different U/X, normalized to 
1 at r = 1 for comparison. Tfie log-log plot reveals a power 
law witfi an interaction dependent exponent rj — 2K. (b) 
gxx (^q — Qyi^ Q_ function of 1/L, demonstrating the absence 
of long-range order. Lines are fits to the form S^^ (g = 0)/L = 
b/L + c/ , with ri taken from (a). Data in (a) and (b) are 
extrapolated to = oo. (c) Luttinger liquid parameters K 
and V. K is extracted from fits to the form S^^{r) — a/r^ 
shown as lines in (a), v is estimated from the density structure 
factor N{q — 2h/L,t) and finite-size extrapolation; wp = 2A. 



is left. The quadratic Hamiltonian ^ gives 

1 



^^^(x) ^ (S'^(a;)S'^(0)) 



2K 



cos(2A:Fa^) 



S'-\x) = {S'{x)S''{0)) ^ — 

x'^ 

N{x) = {n{x)n{Q)) - \ 



(3) 



The transverse correlator S^^{x) involves spin- flip 
scattering and hence picks up 2k-p oscillations in addition 
to an interaction dependent power-law exponent. N{x) 
and S'^^(a;) involve scattering processes only within the 
left or right movers, and retain their Fermi liquid form. 
For K <1, transverse spin correlations dominate. 

For U — Q {K = 1), Eq. (|3| can be verified by explic- 
itly solving the effective model of Eq. p}. For f/ > 0, 
we calculate the correlation functions pi exactly using 
QMC, and we show 5^^(r) at half fiUing in Fig. [TJa). 
With increasing C//A, we observe a progressively slower 
decay, corresponding to a reduction of the exponent 2K 
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FIG. 2; (Color online) Doping dependence of spin and charge 
structure factors for (//A = 8 from grand-canonical CTQMC 
simulations (L = 32, /3 = 60). The q = Q points in (b) and 
(c) are linear extrapolations. 



[see Eq. ([s])]. Similar behavior was observed in numerical 
work (Qjill^. The finite-size scaling of the structm-e factor 
S^^{q = 0) shown in Fig. [ijb) confirms the absence of 
long-range, ferromagnetic order even for U/X — 40. This 
conclusion is based on simulations of very large systems 
(up to 24 X 64), whereas smaller sizes would incorrectly 
suggest long-range order [TTJ[TH]. True (Ising) long-range 
order becomes possible (at T = 0) if the C/(l) symmetry 
of the KMH model is further reduced to Z2 by Rashba 
coupling. On the mean-field level, symmetry breaking 
occurs for any U > due to a logarithmic instability. 

Figure [ijc) shows the LL parameter K, as obtained 
from rj = 2K [Eq. and the renormalized velocity 

V calculated from N{q,T) (see below), v closely follows 
the Fermi velocity v-p, with a small offset independent 
of A. This result, consistent with v being inherited from 
the bulk and slightly renormalized by U, conflicts with 
Eq. We find < 1/2 for the values of U/X consid- 
ered, far from the noninteracting limit K = 1 and in the 
regime where umklapp scattering is relevant |51 llll I18j . 
However, for translationally invariant systems, this term 
is allowed only for the special case of exactly half filling 
and in the presence of Rashba coupling. 

The cos(2fcF?') contribution to S^^{r) becomes appar- 
ent upon doping the system by varying the chemical po- 
tential inside the bulk gap. As shown in Fig. l2Fa) for 
U/X = 8, the peak in the structure factor S'^^ (q) moves 
from g = to finite q with increasing doping, reflect- 
ing the change of fcp. In contrast, the spin-z and charge 
structure factors retain their cusp structure in the long- 
wavelength limit related to the decay in real space. 
Deviations from Eq. ([s]) are observed for S^^{q) outside 
the long- wavelength limit for large U/X> 10, and can be 
related to the strong-coupling effects discussed below. 

Figure [3] shows the spectral function A^{k,Ljj). For 




FIG. 3: (Color online) Single-particle spectral function 
A-^{k,uj) from projector-CTQMC simulations for L = 24, 
e = 80 and (a) U/X = 8, (b) U/X = 40. 



weak coupling [U/X = 8, Fig. [sj^a)], we see a dominant 
linear mode crossing the Fermi level, and the spectrum 
closely resembles the case U = 1\. With increasing 
correlations [U/X = 40, Fig. [3]jb)], spectral weight is 
suppressed at low energies in favor of new high-energy 
features. The spectral function in Fig. ^h) is funda- 
mentally different from Fig. [sj^a) and previous results 
[3]. The latter were qualitatively fully captured by the 
HTL model which gives a branch cut {TZ^x, ^)7^(0, 0)) - 

{x — vt)^^^'^^^^'^K These additional features arise only 
in the limit of small spin-orbit coupling (large U/X S> 1). 

The HTL model ^ does not account for the single- 
particle spectrum in Fig. ^h) . Except for the bulk gap at 
the Dirac points, the locus of spectral weight at high ener- 
gies near kp bears remarkable resemblance to the zigzag 
edge states of graphene ribbons with U > [TSj. The 
physics of the latter is dominated by quasi-long-range 
transverse spin fluctuations at the edge. Similarly, a very 
slow decay of transverse spin fluctuations at large U/X 
[Fig. [TJa)] is a generic signature of the strong-coupling 
regime of the KMH model, see also Refs. [9j ITT]. 

To identify spin-flip scattering as the physical ori- 
gin of the high-energy features of Fig. ^h), we com- 
plement our numerical results by a simple yet suffi- 
cient analytical approximation. Rewriting the Hub- 
bard term as Hu = -^U J2q {Sq + S+) with 

~ L^^^'^ J^k'^lf'^k+qi, perturbation theory to order 
U'^ gives the self-energy 



T,^{k,iuJm) 



El 

I3L 



X! Xo{Q,'i^m)Gj^{k-q,iuJr, 

q.iQjn 



with inelastic spin-hip scattering off q = 2fcF trans- 
verse spin fluctuations described by the susceptibil- 
ity XoiqJ^r.i) = -{PL)-'^Y.k,iujr^Gl{k + q,iuj„, + 

i^m) Go(fc, ioJm)- The results for A^{k, oj) shown in Fig. [4] 
qualitatively reproduce the numerical data in Fig. [Sj For 
U/X = 40, the linear low-energy mode is better visi- 
ble than in the numerical spectrum whose calculation 
involves analytical continuation. The emergence of high- 
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FIG. 4: (Color online) Single-particle spectral function 
A^{k,Lo) from second-order perturbation theory. U/t = 2. 
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FIG. 5: (Color online) Luttinger parameter K (from Fig. [T]) 
and spectral weight Z from fits to the density structure factor 
N{q, t) at 6 = 80 and finite-size scaling. The Luttinger liquid 
result K = Z is violated for a wide range of parameters. 

energy features predominantly above the bulk gap with 
increasing U/X causes, via the sum rule / dujA„{k,ui) = 
1, a depletion of spectral weight at low energies. Spin- flip 
scattering is present for any U > 0, but its effects become 
apparent in A^{k,uj) when U/X^ 1. 

The deviations from the HTL model and their effect on 
edge transport may be quantified by measuring the spec- 
tral weight Z of low-energy particle-hole excitations using 
the dynamic charge structure factor N(q,uj). Fitting the 
linear mode to the form N{q,uj) — ^ZqS{uj — vq), with 



weight Z and velocity v estimated from N{q = 2tt/L,t), 
we can use the continuity equation to obtain the Drude 
weight D = Zv. This result can be compared to the re- 
lation D = Kv following from Eq. implying Z = K 
for the HTL in accordance with results at [/ = (not 
shown). However, as apparent from the QMC results in 
Fig.jsj Z ^ K over a wide range of A/[/, and Z — K holds 
only asymptotically in the weak-coupling limit. Hence, 
irrespective of the interaction strength, inelastic spin-flip 
scattering mediated by bulk states leads to deviations 
from the predictions of the HTL model ([2]). This ef- 
fect gains in magnitude with increasing magnetic corre- 
lations (increasing U /A). The suppression of Z (but with 
Z = K) with increasing U/X (see also Ref. [9]) can be 
understood in the framework of LL theory. Interaction 
effects beyond the model ([2| are reflected in the pro- 
nounced quantitative difference between Z and K. 

Conclusions. — Using quantum Monte Carlo simula- 
tions, we have investigated the validity of the low-energy 
Tomanaga-Luttinger model for edge states of a Z2 topo- 
logical insulator. We found the expected interaction and 
doping dependence of spin and charge correlations, with 
dominant transverse spin fluctuations but no long-range 
order. However, the fact that helical edge modes can- 
not strictly be energetically separated from the insulating 
bulk has important consequences. The bulk states pro- 
vide phase space for inelastic spin-flip scattering. In the 
weak-coupling limit, this leads to minor but quantiflable 
violations from bosonization predictions. For strong cou- 
pling, these scattering processes transfer spectral weight 
from low to high energies in the single-particle spectral 
function, and thereby give rise to features reminiscent of 
graphene zigzag ribbons. 
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